Optimal Bounds for Estimating Entropy with PMF Queries

Let p be an unknown probability distribution on [n] := {1, 2, . . . n} that we can access via two kinds of queries: A SAMP query takes no input and returns x ∈ [n] with probability p[x]; a PMF query takes as input x ∈ [n] and returns the value p[x]. We consider the task of estimating the entropy of p to within ±∆ (with high probability). For the usual Shannon entropy H(p), we show that Ω(log n/∆) queries are necessary, matching a recent upper bound of Canonne and Rubinfeld. For the Rényi entropy Hα(p), where α > 1, we show that Θ(n 1−1/α/2∆) queries are necessary and sufficient. This complements recent work of Acharya et al. in the SAMP-only model that showed O(n1−1/α) queries suffice when α is an integer, but Ω̃(n) queries are necessary when α is a noninteger. All of our lower bounds also easily extend to the model where CDF queries (given x, return ∑ y≤x p[y]) are allowed.